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ABSTRACT 

The aim of this research study is to investigate the articles published between 2003-2007 in Turkish Online 
Journal of Educational Technology (TOJET) in order to reveal the trends and issues addressed in this electronic 
journal for the field of educational technology. The specific purposes of this article are to reveal: (1) the 
methodologies and theories that underlie research, (2) the topics that have been most and least heavily 
researched, (3) the design types that shapes research, (4) the issues on selection, size and level of the sample, 
and (5) some other issues like number of authors, data collection methods, references and variables. After a 
detailed document analysis, the articles are carefully investigated and categorized according to the selected 
criteria. Then these qualitative data were analyzed inductively, categorized according to already existing or 
emerging themes. Moreover, some parts of the analyzed data are transformed into numerical values and 
presented as graphics. The results showed that in order to improve the quality of research in the field of 
educational technology; research studies should have a theoretical basis, the mixed method of research 
(qualitative and quantitative) should be used to complement each other, the research studies should address K- 
12 as much as Higher Education, new and emerging research topics should be sought, interdisciplinary topics 
should be investigated, and diversity in terms of sample selection, data collection, and research design should be 
sought. 
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INTRODUCTION 

A recent review of the literature discloses an abundance of educational technology research studies employing a 
variety of research methodologies in a variety of educational settings. The research in this area has evolved 
especially since the Internet and communication technologies became widespread. Thus, the early studies are 
mainly focused on the impact of technology on the students in terms of cognitive and affective outcomes 
academic achievement. Several meta-analyses have investigated the impact of computer-assisted instruction on 
student outcomes (Waxman & Michko, 2003). Some other meta-analyses have examined aspects such as the 
effects of microcomputer applications in elementary schools (Ryan, 1991) and the effects of computer 
programming on student outcomes (Liao & Bright, 1991). Niemiec and Walberg (1992) summarized the findings 
on 13 quantitative research syntheses that were conducted between 1975 and 1987 and found that the average 
effect size was .42, which indicated that the average student who received computer-based instruction scored at 
the 66th percentile of the control group distribution (i.e. the 50th percentile). Overall, these meta-analyses have 
documented the positive effects of educational technology on student achievement in general (Schacter, 2001; 
Sivin-Kachala, 1998; Wenglinsky, 1998). 

On the other hand, Blok, Oostdam, Otter and Overmaat (2002), investigated the effectiveness of computer- 
assisted instruction (CAI) programs in supporting beginning readers. Their findings were similar to earlier meta- 
analyses by Kulik and Kulik (1991) and Ouyang (1993), which also examined the effects of CAI and found it to 
have positive but small effects (Waxman & Michko, 2003). Lou, Abrami and d’Apollonia (2001) examined the 
effects of students working in a small group versus working individually when students were using computer 
technology. They found that small-group learning had more positive effects than individual learning. Other 
recent meta-analyses in technology have examined topics such as the effectiveness of interactive distance 
education (Cavanaugh, 2001), computer-assisted instruction in science education (Bayraktar, 2001-2002), and 
computer-based instructional simulation (Lee, 1999). 

Schacter (1999) mentioned that there are analysis of 5 largest scale studies about impacts of Education 
Technology which were selected for their scope, comprehensive samples, and generalizability to local, state, and 
national audiences. The first study (Kulik, 1994) is employed a statistical technique called meta analysis to 
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aggregate the results over 500 individual studies to draw a single conclusion. This study summarized more than 
97 of the computer-based instruction studies conducted in the 1980s, noting that students typically learn more 
and faster in courses involving computer-based instruction and have more positive attitudes. The results of his 
meta-analysis support the use of computers as a means to improve student achievement. The second (Sivin- 
Kachala, 1998) reviewed hundreds of individual studies whereby the authors shed light on consistent patterns 
that emerged across studies. The third (Baker, Gearhart & Herman, 1994) reviewed a partnership between Apple 
and five schools across the nation. The fourth study (Mann, 1999) reported the results of West Virginia’s 10 year 
statewide education technology initiative. The fifth (Wenglinsky, 1998) assessed a national sample of fourth and 
eight grade students using simulation and higher order thinking technologies. The sixth (Scardamalia & Bereiter, 
1996) and seventh (Harel & Papert, 1991) and (Harel, 1990) reviewed two smaller scale studies that show the 
promise of never emerging technologies on student learning. 

On the other hand, educational leaders invited the meeting of Preparing Tomorrow's Teachers to Use 
Technology and they agreed on the need for identification through research of the best practices in the use of 
technology in teacher education. Studies to determine the generalizable effects of technology in teacher 
preparation programs are essential because of the key role of the teacher in education and because of the existing 
evidence on the need for in-depth preparation of teachers to use technology effectively (Thompson, 2005). In 
addition leading researchers in education widely agree that more theory and evidence based research in 
education is needed (Feuer, Towne, & Shavelson, 2002; Roblyer & Knezek, 2002). 

Hence, this article summarizes the past five years’ studies in educational technology to help us become better 
prepared for future research challenges. This goal is accomplished by examining all articles about educational 
technologies of TOJET between 2003 and 2007. The specific purposes of this article are to reveal: (1) the 
methodologies and theories that underlie research, (2) the topics that have been most and least heavily 
researched, (3) the design types that shapes research, (4) the issues on selection, size and level of the sample, and 
(5) some other issues like number of authors, data collection methods, references and variables. It is expected 
that this article will provide directions and suggestions for future research by revealing the gaps and needs. 

METHOD 
Research Procedure 

For this research study, one of the leading journals in the field of educational technology. The Turkish Online 
Journal of Educational Technology (TOJET) was selected. TOJET is a quarterly, peer-reviewed international 
electronic journal which can be accessed online from the address http://www.tojet.net. 

TOJET is devoted to the issues and applications of educational technology to enhance learning and teaching. 
TOJET is included in many databases like Education Research Index, ERIC and EBSCO Online. It has editors 
and editorial board members from Turkey, TRNC, USA, Germany, Holland, Italy, Finland, Ukraine, Malaysia, 
Canada, India and Jordan. The main purpose of knowledge sharing through TOJET as stated in its web page, is 
the contribution toward the improvement of education and learning through educational technology through 
various ideas and practical solutions. 

Since the scope of the journal was educational technology, all the published manuscripts for the last five years 
(2003-2007) formed the content for this research. Several criteria were preset by the researchers in order to draw 
a general picture of what has been done through five years. The main goal of this content analysis was to find 
answers to some questions. What characterizes research on educational technology? How has research on 
educational technology developed recently? What is the general trend of researchers about educational 
technology? 

Hence, for the content analysis of manuscripts published in TOJET in last five years, researchers set the criteria 
as follows; 

♦ Research Topics 

♦ Authors 

♦ School Level 

♦ Research Theories 

♦ Research Design 

♦ Sample Selection Method 

♦ Sample Size 

♦ Data Collection Methods 

♦ Resources 
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♦ Number of Variables (dependent-independent) 

♦ Research Type (qualitative-quantitative) 

The purpose of the present study is to categorize and synthesize recent research studies published in TOJET for 
the last five years, in order to shed light on future studies that will be conducted in the field of educational 
technology. Thus, this study reports on the results of the content analysis of TOJET, to provide guidance for the 
development of an educational technology research agenda. 

Data Analysis 

After a detailed document analysis, the articles are carefully investigated and categorized according to the 
criteria specified below. Then these qualitative data were analyzed inductively, categorized according to already 
existing or emerging themes. Moreover, some parts of the analyzed data are transformed into numerical values 
and presented as graphics. 

FINDINGS 

The findings of this study are reported according to the selected criteria. Some results are presented as numbers, 
whereas some are provided as graphics. 

Research Topics 

Although it was difficult to distinguish between the research topics authors published, an inductive coding 
facilitated the categorization of emerging themes (Table 1). The following research topics were emerged from 
the analyzed data. Since some articles pointed out more than one topic, they were listed more than one times 
under different headings. 



Table 1 Research Topics Emerged from the Document Analysis 



Research Topics Number of Articles 

Effects of Computer Based Instruction/Multimedia 36 

Web-Based/Online Education 35 

Integration of Technology into Education 32 

Assessment/Evaluation of Instructional Software 15 

Perceptions about computers/technology 12 

Design/Development of Instructional Software 12 

Teachers’ Competencies of Technology 10 

Attitudes toward computers/technology 8 

Presentation of Real Applications 8 

Alternative Assessment 6 

Constructivist Learning Environments 6 

Cognitive Styles/Tools 5 

Benefits/Challenges 5 

Effects on Student Achievement 5 

Mobile Learning 4 

Comparison of traditional vs. online learning 3 

Virtual Reality 3 

Management Issues 3 

Creativity 2 



The research topics which appeared one times during the analysis was; intelligent tutoring, visual learning 
strategies, creativity, interactions, e-readiness, motivation, and self-efficacy. Top three topics which also lead the 
research in the field are composed of research on computer based and online technologies, and their integration 
into education. 

Number of Authors 

Most of the articles have one or two authors which can be explained as the lack of collaboration between 
academicians (Table 2). Another main reason maybe the academic carrier requirements that most of the authors 
should meet in terms of points from these research studies. 




The Turkish Online Journal of Educational Technology - TOJET April 2009 ISSN: 1303-6521 volume 8 Issue 2 Article 12 



Table 2 Author numbers per articles 

Number of Authors Number of Articles 



1 

2 

3 

4 

5 

6 
10 



87 

59 

26 

10 

3 

2 

1 



School Level 

It is understood from the graph that except 2003 and 2005, in the other three years higher education is more 
preferable for authors (Figure 1). In 2003 more KI2 schools and in 2005 equal amount of K12 and HE has been 
studied. The reason why HE is much preferred than K12 by researchers may be the difficulty of getting 
permission, in other words the bureaucracy. 



Figure 1 Number of School Levels Per Year 
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Research Theories/Framework 

Among 187 articles only 76 based their discussion or findings on theoretical basis. In 2003, “computer aided 
instruction” is the most favored one (Yeditepe & Karadag, 2003; Alako 9 , 2003; Altun, 2003; Ak 9 ay et al., 2003; 
Giiven & Karata§, 2003; Ipek, 2003; Kocasara 9 , 2003; Morgil et al., 2003; Arslan, 2003; ^ekba§ et al., 2003; 
Yenice, 2003; Boynak, 2003). The underlying theories in these articles are summarized in Table 2. 



Table 2 Research Theories/Framework 



Research Theories/Framework 

Computer Aided/Based Instruction 
Web-Based Education 
Constructivist Learning Environments 
Diffusion of Innovation Theory (Rogers) 
Social Learning Theory (Bandura) 
Multiple Intelligences Theory (Gardner) 
Interactive/ Active Learning 
Cognitive Learning Theories 
Other Theories 
Total 



Number of Articles 

21 

16 

8 

6 

3 

3 

3 

4 
12 
76 



The field of other theories/framework includes problem-based learning, visual learning, creative thinking, 
inquiry based learning, Gibson’s model of affordances, generative theory, individual learning, mobile education, 
experiential learning, cooperative learning, virtual reality, and item response theory. It is obvious that theoretical 
preferences heavily based on learning theories. 
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Research Design 

The analysis indicated that most of the studies are in the type of literature review (74) and descriptive (63) (Table 
3). The number of experimental studies is very low that explains the difficulties of experimental studies. The 
results also illustrates that the last two years, namely 2006 and 2007, have a few literature review and more 
descriptive and experimental studies. In 2006 and 2004, the number of experimental studies are higher than the 
others (Qetin & et ah, 2004; Ozsoy & Yildiz, 2004; Morgil & et al., 2004; Baki & Birgin, 2004; Morgil & et al., 
2004; Diindar & Kiyici, 2004; Baki & §ahin, 2004; Ozdener & Saym, 2004; Aksoy & Ozad, 2004; Ak9ay & 
et al, 2006; §en & Neufeld, 2006; Qataloglu, 2006; Aydm, 2006; Tezci & Dikici, 2006; Yildirim, 2006; Tosun & 
et al., 2006; Gonen & et al., 2006). 



Table 3 Research Design Types Per Year 



Reserach Design 


2003 


2004 


2005 


2006 


2007 


Total 


Descriptive 


17 


14 


16 


6 


10 


63 


Experimental 


7 


9 


6 


9 


3 


34 


Literature Review 


19 


31 


22 


1 


1 


74 


Discussion 


2 


0 


3 


11 


0 


16 


Total 


45 


54 


47 


27 


14 


187 



Sample Selection 

Because of not mentioned directly, most of sample selection types has been predicted by the authors. The 
number of articles that has been clearly mentioned its sample selection type is very low (Table 4). Most common 
selection type was “accessible sampling” as assumed by authors. Furthermore, 89 articles those are mostly in the 
type of discussion or literature review, and presented as “none” in the results. 



Table 4 Sample Selection Method Per Year 



Sample Selection 


2003 


2004 


2005 


2006 


2007 


Total 


Accessible 


10 


15 


18 


14 


7 


64 


Clustered Sampling 


9 


5 


5 


0 


3 


22 


Randomly 


4 


3 


1 


1 


3 


12 


None 


22 


31 


23 


12 


1 


89 


Total 


45 


54 


47 


27 


14 


187 



Sample Size 

The highest percentages of preferable range of sample size are over 200 (Akpmar, 2003; Can, 2003; Olkun & 
Altun, 2003; Tor & Erden, 2004; Ayman & Serim, 2004; Akpmar, 2004, Toprak9i, 2005, Varank, 2005; Arnas, 
2005; Slier & et al., 2005; Deniz, 2005; Ozdemir, 2006; Kabadayi, 2006; Kumtepe, 2006; K09 & Ferneding, 
2007; Baloglu, 2007; Bozkaya & Aydm, 2007; Toprak9i, 2007). The results show that most of the authors can 
reach high numbers of sample size (Table 5). 



Table 5 Sample Sizes Per Year 



Sample size 2003 

1.-29 2 

30-59 6 

60-89 7 

90-119 0 

120-159 5 

160-200 0 

200-> 3 



2004 2005 2006 2007 

5 3 12 

2 7 5 1 

5 12 1 

14 2 2 

4 2 2 1 

2 2 2 2 

3 4 3 4 



Data Collection Methods 

The articles are analyzed in terms of data collection methods also. The methods for collection of data show 
diversity especially in terms of the selected research methods. Although in some research studies more than one 
data collection instrument is used, they are presented separately in Table 6. Figure 2 illustrates the general 
distribution of preferred data collection methods for the last five years. 
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Table 6 Data Collection Methods Per Year 



Data Collection 2003 

Survey 8 

Scale 10 

Interview 5 

Rubric 1 

Achievement Tests 4 
Observation 2 

Portfolio 0 

None 20 

Number of Articles 47 



2004 


2005 


2006 


2007 


2 


7 


0 


0 


12 


10 


9 


11 


4 


6 


1 


1 


0 


1 


1 


0 


5 


4 


6 


0 


3 


3 


0 


1 


1 


0 


0 


0 


30 


23 


12 


1 


54 


47 


27 


14 



Figure 2 Data Collection Types for the last five years 




Since most of them are discussion or literature review, totally 86 researches have no any explanation about the 
type of data collection. Most of the studies collect data by asking scales (52 articles) and the other three main 
parts of the data collections are achievement tests, survey and interview. Although rubric is commonly preferred 
as an alternative assessment tool in last years, it has not been applied in these last five year studies of TOJET. 

Resources 

The articles analyzed in terms of references as being national or international (Figure 3). The results showed that 
although international resources are preferred mostly, the number of national resources used as references are 
increasing year by year which is a satisfactory result. 



Figure 3 Number of References as National or International in the Articles 
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Dependent vs. Independent Variables 

The findings showed that totally 62 articles have one dependent variable, whereas 95 articles have no dependent 
variable (Table 7 and Table 8). Most of these 95 articles are in the type of discussion or literature review. The 
highest number of dependent variables presented in the articles is 5. Since the authors did not mention about the 
variables in their study in detail, most of the variables has been estimated by the authors. The number of articles 
that has been clearly mentioned detailed information about the variables researched was very low. In addition 51 
articles over 186 has one independent variable and the highest number of dependent variables was 8. 



Table 7 Data Collection Methods Per Year 





Number of Dependent Variables 


Years 


1 


2 


3 


4 


5 


None 


2003 


15 


3 


4 






23 


2004 


18 


4 


1 






31 


2005 


11 


5 


3 


1 


1 


26 


2006 


9 


6 








12 


2007 


9 


1 






1 


3 


Total 


62 


19 


8 


1 


2 


95 



Table 8 Data Collection Methods Per Year 





Number of Independent Variables 


Years 


1 


2 


3 


4 


5 


6 


7 


8 


None 


2003 


8 


6 


2 


2 


2 


1 






23 


2004 


15 


2 


2 


2 






1 


1 


31 


2005 


10 


8 


2 












26 


2006 


12 


2 




1 










12 


2007 


6 


2 






1 


1 


1 




3 


Total 


51 


20 


6 


5 


3 


2 


2 


1 


95 



Trend of Quantitative vs. Qualitative Studies 

Most of the mentioned qualitative studies are either literature review or discussion. Until 2005 more literature 
review or discussion types of articles has been studied. In 2006 and 2007 more quantitative studies has been 
conducted. 



Figure 4 Trend of Qualitative vj. Quantitative Methods 




As it is obvious from the graph that the research community has relied heavily on qualitative methods (Figure 3) 
for the first three years, then there is a reverse action. Apart from 2007, it can be concluded that both types of 
research methods are preferred by researchers in a consistent manner. 
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DISCUSSION AND CONCLUSION 

Education systems come across with technologies like computers, web and Internet approximately in last 15 
years, and gained acceleration especially in last ten years in Turkey. The analysis of articles published in TOJET 
in last five years indicated some recent changes of research interests but also showed trends that have remained 
steady in the research community. However, it should be noted that these results are based on 187 articles of 
only one electronic journal. The main findings of this study are summarized according to the previously 
specified criteria as follows. 

1. Research Topics 

The results indicated that the most favored topics which also lead the research in the field are composed of 
research on computer based and online technologies, and their integration into education. Unfortunately, there 
are so few research studies conducted about the recently new emerging technologies like virtual reality and 
mobile learning. When compared with the international studies conducted in other countries, this situation is an 
important shortcoming. However, without necessary investments to technology and people, it is difficult to 
expect to produce research studies parallel with the international literature. 

2. Authors 

The reason why most of the articles have one or two authors is dedicated to the lack of collaboration between 
academicians or the difficulty of meeting academic carrier requirements. Of course this is a cultural phenomenon 
since the academicians in Medical schools are publishing articles with many authors all the times. This 
difference between research fields is interesting and needs further investigation. 

3. School Level 

Although these studies are conducted by researches in universities, more studies are conducted in Higher 
Education than K12 except 2003. In terms of number of students and teachers, when K-I2 and Higher Education 
is compared, it is obvious that more research studies should be conducted in K-12. When limitation to access to 
technology, inadequate skills of teachers and inadequate opportunities of technology training in K-12 and 
difficulties for official authorization are considered, this emerging result is inevitable. 

4. Research Theories 

When theoretical basis of research studies are analyzed, again a shortcoming appears in terms of underlying 
theories which guides research. Among many teaching and learning theories and models which can be adapted to 
technology-rich learning environments, only few of them are addressed by the researchers. Why the researchers 
miss the theoretical bases for their research studies, although the most important part of any research is the 
theory that the study underlines, is a hard question to answer. One of the reasons why there are so few articles 
considered theoretical basis may be related to the youth, in other words experience of the field (Webster & 
Watson, 2002). 

5. Research Design 

The findings revealed literature review and descriptive type of articles are much more than other design types. 
Especially the number of experimental studies is very low. This result may be explained as the difficulties of 
conducting research studies like longitudinal, quasi-experimental and experimental. 

6. Sample Selection Method 

Popularity of “accessible sampling” in sample selection method is another interesting result of this research. 
Among many sample selection methods only two “random” and “clustered” are used as different methods other 
than “accessible” sampling in the whole 187 articles. Bureaucratic procedures seems to result with loose of time, 
money and effort, which discourages researchers to administer their researches in different environments by 
using different methods. 

7. Sample Size 

The findings revealed that there are no problems in terms of sample sizes, since researchers can reach huge 
numbers whenever necessary. 

8. Data Collection Methods 

Compared to the other criteria, there was diversity in data collection methods used in research studies. But in 
fact, many other emerging and different methods should be used for different purposes like triangulation. 
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9. Resources 

Since Turkey is a developing country, most of the other countries are one step further than our country in terms 
of educational technology applications and research. This clarifies why researchers use international references 
more than national ones. On the other hand, increasing number of national references may be dedicated to the 
increase of quality of research in our country. 

10. Number of Variables (dependent-independent) 

Not seeing enough details about the variables was another disappointing result of this study. It was really 
difficult from the content to eliminate the variables and types. 

11. Research Type (qualitative-quantitative) 

In terms of research type, results in general indicated that both types of research methods are preferred by 
researchers in a consistent manner. 

As a summary, it was difficult to analyze the data needed for this article, since the flow and content of the 
articles are showing a huge diversity. Some articles do not meet the criteria which are taught at a compulsory 
course for all graduate students in all departments of the faculty of education. Any way, the results are valuable 
and show us the needs and gaps to fulfill in the future studies. 

Hence, keeping these results in mind, researchers should consider the following points in their research in the 
field of educational technology while guiding their future research studies. 

• The research studies should have a theoretical basis 

• The mixed method of research (qualitative and quantitative) should be used to complement each other 

• The research studies should address K-12 as much as Higher Education 

• New and emerging research topics should be sought 

• Interdisciplinary topics should be investigated 

• Diversity in terms of sample selection, data collection, and research design should be sought 

Since these findings are valid only for the articles of TOJET published in the last five years, the results of this 
research study cannot be generalized. However, the findings may be used to increase the quality of future 
research studies, by taking attention of researchers to important points and gaps realized throughout this research 
study. 
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